Expand description

Pure Rust, high performance implementation of LZ4 compression.

A detailed explanation of the algorithm can be found here.

Overview

This crate provides two ways to use lz4. The first way is through the frame::FrameDecoder and frame::FrameEncoder types, which implement the std::io::Read and std::io::Write traits with the lz4 frame format. Unless you have a specific reason to the contrary, you should only use the lz4 frame format. Specifically, the lz4 frame format permits streaming compression or decompression.

The second way is through the decompress_size_prepended and compress_prepend_size functions. These functions provide access to the lz4 block format, and don’t support a streaming interface directly. You should only use these types if you know you specifically need the lz4 block format.

Example: compress data on stdin with frame format

This program reads data from stdin, compresses it and emits it to stdout. This example can be found in examples/compress.rs:

use std::io;
let stdin = io::stdin();
let stdout = io::stdout();
let mut rdr = stdin.lock();
// Wrap the stdout writer in a LZ4 Frame writer.
let mut wtr = lz4_flex::frame::FrameEncoder::new(stdout.lock());
io::copy(&mut rdr, &mut wtr).expect("I/O operation failed");
wtr.finish().unwrap();

Example: decompress data on stdin with frame format

This program reads data from stdin, decompresses it and emits it to stdout. This example can be found in examples/decompress.rs:

use std::io;
let stdin = io::stdin();
let stdout = io::stdout();
// Wrap the stdin reader in a LZ4 FrameDecoder.
let mut rdr = lz4_flex::frame::FrameDecoder::new(stdin.lock());
let mut wtr = stdout.lock();
io::copy(&mut rdr, &mut wtr).expect("I/O operation failed");

Example: block format roundtrip

use lz4_flex::{compress_prepend_size, decompress_size_prepended};
let input: &[u8] = b"Hello people, what's up?";
let compressed = compress_prepend_size(input);
let uncompressed = decompress_size_prepended(&compressed).unwrap();
assert_eq!(input, uncompressed);

Feature Flags

  • safe-encode uses only safe rust for encode. enabled by default
  • safe-decode uses only safe rust for encode. enabled by default
  • checked-decode will add additional checks if safe-decode is not enabled, to avoid out of bounds access. This should be enabled for untrusted input.
  • frame support for LZ4 frame format. implies std, enabled by default
  • std enables dependency on the standard library. enabled by default

For maximum performance use no-default-features.

For no_std support only the block format is supported.

Modules

LZ4 Block Format

LZ4 Frame Format

Functions

Compress all bytes of input.

Compress all bytes of input into output. The method chooses an appropriate hashtable to lookup duplicates and calls compress_into_with_table. output should be preallocated with a size of get_maximum_output_size.

Compress all bytes of input into output. The uncompressed size will be prepended as a little endian u32. Can be used in conjunction with decompress_size_prepended

Decompress all bytes of input into a new vec.

Decompress all bytes of input into output. output should be preallocated with a size of of the uncompressed data.

Decompress all bytes of input into a new vec. The first 4 bytes are the uncompressed size in litte endian. Can be used in conjunction with compress_prepend_size